Softmax function

Interpretation

• quora:What does the term “soft-max” mean in the context of machine learning?

Attention Is Off By One elucidates what softmax function does in the context of the Transformer model.

Usages

• Softmax Regression
• Hierarchical softmax
• Transformer model

Variations

• Sparsemax